Layer-Wise Adaptive Model Aggregation for Scalable Federated Learning

In Federated Learning (FL), a common approach for aggregating local solutions across clients is periodic full model averaging. It is, however, known that different layers of neural networks can have degree discrepancy the clients. The conventional aggregation scheme does not consider such difference and synchronizes whole parameters at once, resulting in inefficient network bandwidth consumption. Aggregating are similar make meaningful training progress while increasing communication cost. We propose FedLAMA, layer-wise adaptive scalable FL. FedLAMA adjusts interval manner, jointly considering This fine-grained strategy enables to reduce cost without significantly harming accuracy. Our extensive empirical study shows that, as increases, remarkably smaller accuracy drop than aggregation, achieving comparable efficiency.